Functions in Julia with more image processing and transformations
Learning about Julia funtions by implementing a edge detection filter to an image
In this write up
- write function to allow user to input a kron factor, scaling value (and also convex optimisation)
- Downsampling, upsampling,
- Offset index
- More image processing using custom function for edge detection and ImageFiltering's imfilter..
- N0f8/16/32/64 data type
begin
using Images, Colors, ColorVectorSpace, ImageShow, FileIO, ImageIO
using ImageFiltering
using OffsetArrays
using Plots
end
pexels = load("./images/pexels-photo-2422290.jpeg")
size(pexels)
Downsampling
By taking only some pixels of the image, we get a lower resolution of the original image (reducing storage size).
pexels_downsample = pexels[1:3:end, 1:4:end]
size(pexels_downsample)
The original image which was of size 750x1042 has now been reduced to 250x261 by taking every 3rd pixel of the rows and every 4th pixel of the columns.
Upsampling is the inverse and will just add more pixels to an image. How to upsample is the interesting bit, it is not as simple as just picking which pixel to keep, however there are many ways of upsampling.
We can upsample an image by multiplying the matrix array by a square matrix using the Kronecke product. A matrix of mxn multiplied by a square matrix of pxp will result in an (mxp)x(nxp) array which is the upsampled image.
In Julia, this can be achieved by using the kron function, whose arguments are the two arrays, which in our case are the image and a square matrix of 1s.
pexels_upsample = kron(pexels_downsample, fill(1, 3, 4))
fill(1, 3, 4)
size(pexels_upsample)
3 ways of writing functions:
- The short form
f(a) = a * a
- The anonymous form
x -> sin(x)
- The block form
function func(x, y)
x + y, |x - y|end
(return is optional)
Writing a function that will pass 2 arguments to a function, m and n dimensions of a matrix of 1s that will be used to kron upsample an image.
Notes:
- dim1 is a keyword argument, used as a label (separated from the unlabelled arguments with a semi-colon)
- n will default to dim1 if only one additional argument to image is specified, creating a square matrix.
function upsample(image; dim1=m, n=dim1)
kron(image, fill(1, dim1, n))
end
methods(upsample)
The methods function gives more information about a particular function
cat = load("./images/cat.jpg")
upsample(cat, dim1=3, n=4)
upsample(cat, dim1=5)
Scaling is a linear transformation in which a scalar multiplies the image.
c = 0.5
c .* cat
#the dot is indicating that the scalatr is being broadcast to every pixel of the image
The cat picture has visibly become dull.
0.01 .* cat
3 .* cat
As c goes to 0, the picture becomes darker, whereas as c goes beyond 1, the picture saturates...
Combining two or more images, by simply adding the arrays of the images together.
upside_cat = cat[end:-1:1, :]
mixed_cat = cat + upside_cat
Convex combination where α and β used to scale the two mixed images add up to 1
.5 .* cat + .5 .* upside_cat
Write a function that takes α and does a convex transformation of two provided images
function ConvexTransform(image1, image2, α)
α .* image1 + (1 - α) .* image2
end
ConvexTransform(pexels, pexels[end:-1:1, :], 0.39)
Many image processing functions use grayscale images, converting to grayscale in Julia...
colorview(Gray, Gray.(cat))
edge_detect = [0 -1 0; -1 4 -1; 0 -1 0]
We want to convolve the cat image with the edge detect kernel. Moving the kernel across the pictures, multiplying elementwise and returning the sum, which then becomes the new value of the pixel.
function EdgeDetect(image, filter; padding=0)
img_gray = channelview(Gray.(image))
temp = fill(0N0f8, height(img_gray)+2*padding, width(img_gray)+2*padding)
temp[padding+1:end-padding, padding+1:end-padding] = img_gray
convolved = fill(0.0, height(img_gray)-height(filter)+1+2*padding, width(img_gray)-width(filter)+1+2*padding)
for i in 1:1:height(temp)-height(filter)+1
for j in 1:1:width(temp)-width(filter)+1
a = sum(temp[i:i+height(filter)-1, j:j+width(filter)-1] .* filter)
convolved[i, j] = a
end
end
return colorview(Gray, convolved)
end
The function takes as arguments, an image, a filter and a padding value (defaults to 0) and is implemented as follows:
- Convert the image to grayscale
- Create temporary array of zeros with added rows and columns to represent padding around the image.
- To the temporary array, superimpose the gray image, by replacing the 0s with values from the gray image leaving 0s at the outer rows and columns, the padding.
- Create another temporary array, whose values will be replaced with the new pixel values as the filter moves across the image.
- Return the filtered image
EdgeDetect(cat, edge_detect, padding=1)
leo = load("./images/leo.jpg")
EdgeDetect(leo, edge_detect)
Or you could just use the imfilter function from the ImageFiltering package...
imfilter(cat, edge_detect)
function FilterImage(image, filter; padding=0)
img_array = reshape(channelview(image), height(image), width(image), 3)
temp = fill(0N0f8, height(img_array)+2*padding, width(img_array)+2*padding, 3)
temp[padding+1:end-padding, padding+1:end-padding, :] = img_array
convolved = fill(0.0, height(image)-height(filter)+1+2*padding, width(image)-width(filter)+1+2*padding, 3)
for k in 1:1:3
for i in 1:1:height(temp)-height(filter)+1
for j in 1:1:width(temp)-width(filter)+1
a = sum(temp[i:i+height(filter)-1, j:j+width(filter)-1, k] .* filter)
convolved[i, j, k] = a
end
end
end
return colorview(RGB, reshape(convolved, (3, height(convolved), width(convolved))))
end
Trying out other filters using imfilter..
sharpen = [0 -1 0; -1 5 -1; 0 -1 0]
imfilter(cat, sharpen)
Built-in filters..
imfilter(cat, Kernel.gaussian(3))
imfilter(cat, Kernel.Laplacian())
Other filters that are available for use with imfilter
- sobel
- prewitt
- ando3, ando4, and ando5
- scharr
- bickley
- DoG (Difference-of-Gaussian)
- LoG (Laplacian-of-Gaussian)
- gabor
- moffat
Offsetting
Julia can allows arbitrary indexing (instead of starting at 1 (default) or 0 (Python)). Can be very useful in dealing with arrays,especially multi-dimensional ones.
M = collect(1:10)
OffM = OffsetArray(M, -5:4)
println(OffM[-5])
println(OffM[-2])
println(OffM[4])
Off = [1 2 2; 3 5 3; 2 6 5]
OffCentered = OffsetArrays.centered(Off)
OffsetArrays.center(Off)
N0f8/16/...
When processing images, the array can be stored in this format which is a normalised (between 0 and 1) which makes image processing handy. Can be manipulated to result in floats (like mulitiplication and summing in filters or convolutions)
x = fill(0.0N0f8, 3, 3)
x[1, 1]
dump(x[1, 1])
y = channelview(cat)[1, 6, 3]
dump(y)
float(y)